Capstone Project Group A CV1

Pneumonia Detection Challenge

The goal is to build a pneumonia detection system, to locate the position of inflammation in an image.

This model is built using Detectron2 (Faster_RCNN) on pytorch: https://github.com/facebookresearch/detectron2

Data preprocessing steps needed are as below:

  1. Convert all the dicom images to jpg as Detectron2 can't directly read dicom images : convert_to_jpg()
  2. Image augmentation using albumentations library
  3. Convert the data to coco json format : get_data_dicts()
In [1]:
!pip install pyyaml==5.1
!pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
!pip install pydicom
!pip install sklearn
!pip install funcy
!pip install argparse
Requirement already satisfied: pyyaml==5.1 in /usr/local/lib/python3.7/dist-packages (5.1)
Looking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
Requirement already satisfied: detectron2 in /usr/local/lib/python3.7/dist-packages (0.6+cu111)
Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2) (1.1.0)
Requirement already satisfied: hydra-core>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2) (1.1.1)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from detectron2) (0.16.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from detectron2) (3.2.2)
Requirement already satisfied: pydot in /usr/local/lib/python3.7/dist-packages (from detectron2) (1.3.0)
Requirement already satisfied: black==21.4b2 in /usr/local/lib/python3.7/dist-packages (from detectron2) (21.4b2)
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from detectron2) (0.8.9)
Requirement already satisfied: fvcore<0.1.6,>=0.1.5 in /usr/local/lib/python3.7/dist-packages (from detectron2) (0.1.5.post20211023)
Requirement already satisfied: iopath<0.1.10,>=0.1.7 in /usr/local/lib/python3.7/dist-packages (from detectron2) (0.1.9)
Requirement already satisfied: pycocotools>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from detectron2) (2.0.2)
Requirement already satisfied: Pillow>=7.1 in /usr/local/lib/python3.7/dist-packages (from detectron2) (7.1.2)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from detectron2) (2.6.0)
Requirement already satisfied: omegaconf>=2.1 in /usr/local/lib/python3.7/dist-packages (from detectron2) (2.1.1)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from detectron2) (1.3.0)
Requirement already satisfied: tqdm>4.29.0 in /usr/local/lib/python3.7/dist-packages (from detectron2) (4.62.3)
Requirement already satisfied: yacs>=0.1.8 in /usr/local/lib/python3.7/dist-packages (from detectron2) (0.1.8)
Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (0.4.3)
Requirement already satisfied: regex>=2020.1.8 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (2021.10.23)
Requirement already satisfied: pathspec<1,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (0.9.0)
Requirement already satisfied: click>=7.1.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (7.1.2)
Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (1.4.4)
Requirement already satisfied: toml>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (0.10.2)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (3.7.4.3)
Requirement already satisfied: typed-ast>=1.4.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2) (1.4.3)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2) (5.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2) (1.19.5)
Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2) (5.2.2)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2) (4.8)
Requirement already satisfied: portalocker in /usr/local/lib/python3.7/dist-packages (from iopath<0.1.10,>=0.1.7->detectron2) (2.3.2)
Requirement already satisfied: cython>=0.27.3 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2) (0.29.24)
Requirement already satisfied: setuptools>=18.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2) (57.4.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2) (1.3.2)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from cycler>=0.10->matplotlib->detectron2) (1.15.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources->hydra-core>=1.1->detectron2) (3.6.0)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (3.17.3)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (1.0.1)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (3.3.4)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (1.41.0)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (2.23.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (1.8.0)
Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (1.35.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (0.37.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (0.4.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (0.6.1)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2) (0.12.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (4.7.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2) (4.2.4)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (1.3.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard->detectron2) (4.8.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard->detectron2) (0.4.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2021.5.30)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2.10)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (3.1.1)
Requirement already satisfied: pydicom in /usr/local/lib/python3.7/dist-packages (2.2.2)
Requirement already satisfied: sklearn in /usr/local/lib/python3.7/dist-packages (0.0)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from sklearn) (0.22.2.post1)
Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->sklearn) (1.4.1)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->sklearn) (1.19.5)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->sklearn) (1.0.1)
Requirement already satisfied: funcy in /usr/local/lib/python3.7/dist-packages (1.16)
Collecting argparse
  Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB)
Installing collected packages: argparse
Successfully installed argparse-1.4.0
In [2]:
# check pytorch installation: 
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())
assert torch.__version__.startswith("1.9")   # please manually install torch 1.9 if Colab changes its default version
1.9.0+cu111 True
In [3]:
# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

# import some common libraries
import numpy as np
import pandas as pd
import glob, os, json, cv2, random
from google.colab.patches import cv2_imshow
import pydicom as pyd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report

# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog

import matplotlib.pyplot as plt
In [4]:
base_dir='/content/drive/MyDrive/colab'
data_dir = base_dir+'/input'
working_dir = base_dir+'/working'

train_labels_df = pd.read_csv(data_dir+'/stage_2_train_labels.csv')
label_meta_data = pd.read_csv(data_dir+'/stage_2_detailed_class_info.csv')

train_dicom_dir = os.path.join(data_dir, 'stage_2_train_images')
test_dicom_dir = os.path.join(data_dir, 'stage_2_test_images')

train_labels_df.tail(10)
Out[4]:
patientId x y width height Target
30217 c1e228e4-b7b4-432b-a735-36c48fdb806f NaN NaN NaN NaN 0
30218 c1e3eb82-c55a-471f-a57f-fe1a823469da NaN NaN NaN NaN 0
30219 c1e73a4e-7afe-4ec5-8af6-ce8315d7a2f2 666.0 418.0 186.0 223.0 1
30220 c1e73a4e-7afe-4ec5-8af6-ce8315d7a2f2 316.0 504.0 179.0 273.0 1
30221 c1ec14ff-f6d7-4b38-b0cb-fe07041cbdc8 609.0 464.0 240.0 284.0 1
30222 c1ec14ff-f6d7-4b38-b0cb-fe07041cbdc8 185.0 298.0 228.0 379.0 1
30223 c1edf42b-5958-47ff-a1e7-4f23d99583ba NaN NaN NaN NaN 0
30224 c1f6b555-2eb1-4231-98f6-50a963976431 NaN NaN NaN NaN 0
30225 c1f7889a-9ea9-4acb-b64c-b737c929599a 570.0 393.0 261.0 345.0 1
30226 c1f7889a-9ea9-4acb-b64c-b737c929599a 233.0 424.0 201.0 356.0 1
In [5]:
def get_files(dicom_dir):
    files = glob.glob(dicom_dir+'/'+'*.dcm')
    return list(set(files))

def parse_dataset(dicom_dir, anns): 
    image_files = get_files(dicom_dir)
    image_annotations = {fp: [] for fp in image_files}
    for index, row in anns.iterrows(): 
        fp = os.path.join(dicom_dir, row['patientId']+'.dcm')
        image_annotations[fp].append(row)
        if index == 0:
          print(row)
    return image_files, image_annotations 

image_files, image_annotations = parse_dataset(train_dicom_dir, anns=train_labels_df)
patientId    0004cfab-14fd-4e49-80ba-63a80b6bddd6
x                                             NaN
y                                             NaN
width                                         NaN
height                                        NaN
Target                                          0
Name: 0, dtype: object
In [6]:
!pip install -q -U albumentations
!echo "$(pip freeze | grep albumentations) is successfully installed"
albumentations==1.1.0 is successfully installed
In [7]:
import albumentations as A
from  albumentations.core.composition import BboxParams
transform = A.Compose([
              A.RandomCrop(width=1024, height=1024),
              A.HorizontalFlip(p=0.5),
              A.RandomBrightnessContrast(p=0.2),
          ],  bbox_params=BboxParams(format='coco', min_area=1024, min_visibility=0.1)
)
In [8]:
def convert_to_jpg(input_file, output_file):
    ds = pyd.dcmread(input_file)
    cv2.imwrite(output_file, ds.pixel_array)

# for d in ["test"]:
#     input_path = os.path.join(data_dir, "stage_2_"+d+"_images")
   
#     for file in get_files(input_path):
#          output_file = os.path.join(data_dir, d, os.path.basename(file).replace(".dcm", "")+".jpg")
#          convert_to_jpg(file, output_file)
In [9]:
def show(d):  
    img = cv2.imread(d["file_name"])
    visualizer = Visualizer(img[:, :, ::-1], metadata=pnuemonia_metadata, scale=0.5)
    out = visualizer.draw_dataset_dict(d)
    cv2_imshow(out.get_image()[:, :, ::-1])
In [10]:
from detectron2.structures import BoxMode
  
def get_data_dicts(img_dir, augment):
    dataset_dicts = []
    for idx, arr in enumerate(image_annotations.values()):
        record = {}
        filename = os.path.join(img_dir, arr[0]["patientId"]+".jpg")
        
        img = cv2.imread(filename)
        if img is None:
          continue;
        height, width = img.shape[:2]
        
        record["file_name"] = filename
        record["image_id"] = idx
        record["height"] = height
        record["width"] = width
      
        objs = []
        aug_boxes = []
        for v in arr: 
          if np.isnan(float(v["x"])):
            continue
          obj = {
                  "bbox": [v["x"], v["y"], v["width"], v["height"]],
                  "bbox_mode": BoxMode.XYWH_ABS,
                  "category_id": 0,
              }
          aug_boxes.append([v["x"], v["y"], v["width"], v["height"], 'pnuemonia']) 
          objs.append(obj)
        record["annotations"] = objs
        # dataset_dicts.append(record)
        
        rndm = random.randint(1, 10)
        if augment and rndm < 3 and len(objs) > 0:
            transformed = transform(image=img, bboxes=aug_boxes)
            transformed_image = transformed['image']
            transformed_bboxes = transformed['bboxes']
            aug_file_name = os.path.join(img_dir,'augment', arr[0]["patientId"]+".jpg")
            cv2.imwrite(aug_file_name, transformed_image)
            # plt.imshow(cv2.cvtColor(transformed_image, cv2.COLOR_BGR2RGB))
            aug_objs = []
            for row in transformed_bboxes: 
              obj = {
                  "bbox": row[0:4],
                  "bbox_mode": BoxMode.XYWH_ABS,
                  "category_id": 0,
              }
              aug_objs.append(obj)

            record_aug = {}    
            record_aug["image_id"] = idx
            record_aug["height"] = height
            record_aug["width"] = width
            record_aug["annotations"] = aug_objs
            record_aug["file_name"] = aug_file_name
            record_aug["image_id"] = idx+40000
           
            show(record)
            show(record_aug)
            print(filename)
            print(aug_file_name)
            break;

    return dataset_dicts
In [12]:
pnuemonia_metadata = MetadataCatalog.get("train")
print( MetadataCatalog.get("train"))
get_data_dicts(data_dir+'/train', True)
Metadata(name='train')
/content/drive/MyDrive/colab/input/train/382db3e8-f756-48c5-8def-e0630634a680.jpg
/content/drive/MyDrive/colab/input/train/augment/382db3e8-f756-48c5-8def-e0630634a680.jpg
Out[12]:
[]

Create dataset without image augmentation

In [13]:
# import json
# out_file = open(os.path.join(data_dir,"coco-full.json"), "w")
# json.dump(get_data_dicts(data_dir+'/train', False), out_file, indent = 4) 
# out_file.close()

Split the dataset into train and validation

In [73]:
with open(os.path.join(data_dir,"coco-full.json"), 'r') as myfile:
    data=myfile.read()

# parse file
obj = json.loads(data)
 
X_train, X_val = train_test_split(obj, test_size=0.1)
print(len(X_train))
print(len(X_val))
24015
2669
In [15]:
def get_dicts(name):
    return X_train if name == 'train' else X_val

for d in ["train", "val"]:
    try:
      DatasetCatalog.register(d, lambda d=d: get_dicts(d))
      MetadataCatalog.get(d).set(thing_classes=["pneumonia"])
    except:
      print('')
In [24]:
from detectron2.engine import DefaultTrainer
global cfg 
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_C4_1x.yaml"))
cfg.DATASETS.TRAIN = ("train")
cfg.DATASETS.TEST = ()
cfg.OUTPUT_DIR = ( base_dir + "/output")
cfg.DATALOADER.NUM_WORKERS = 2
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_R_50_C4_1x.yaml")  
cfg.SOLVER.IMS_PER_BATCH = 2
cfg.SOLVER.BASE_LR = 0.00025 
cfg.SOLVER.MAX_ITER = 500    
cfg.SOLVER.STEPS = []     
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 512  
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1  

def train_model(): 
  os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
  trainer = DefaultTrainer(cfg) 
  trainer.resume_or_load(resume=False)
  trainer.train()

def evaluate(cnt):
  # Inference should use the config with parameters that are used in training
  # cfg now already contains everything we've set previously. We changed it a little bit for inference:
  cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")  # path to the model we just trained
  cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.7   # set a custom testing threshold
  
  result = [];
  ctr = 0;
  for d in X_val:      
      im = cv2.imread(d["file_name"])
      outputs = predictor(im) 
      no_objects = len(outputs["instances"].pred_boxes)

      predicted = 1 if no_objects > 0 else 0
      ground_truth = 1 if len(d["annotations"]) > 0 else 0
      result.append({"id":  os.path.basename(d["file_name"]).replace(".jpg", ""), "ground_truth": ground_truth, "predicted": predicted})
      ctr +=1
      if ctr > cnt:
        break;
  return result
In [74]:
train_model()
[10/31 12:34:29 d2.engine.defaults]: Model:
GeneralizedRCNN(
  (backbone): ResNet(
    (stem): BasicStem(
      (conv1): Conv2d(
        3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
        (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
      )
    )
    (res2): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv1): Conv2d(
          64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
    )
    (res3): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv1): Conv2d(
          256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (3): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
    )
    (res4): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
        (conv1): Conv2d(
          512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (3): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (4): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (5): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
    )
  )
  (proposal_generator): RPN(
    (rpn_head): StandardRPNHead(
      (conv): Conv2d(
        1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (objectness_logits): Conv2d(1024, 15, kernel_size=(1, 1), stride=(1, 1))
      (anchor_deltas): Conv2d(1024, 60, kernel_size=(1, 1), stride=(1, 1))
    )
    (anchor_generator): DefaultAnchorGenerator(
      (cell_anchors): BufferList()
    )
  )
  (roi_heads): Res5ROIHeads(
    (pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
      )
    )
    (res5): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
        (conv1): Conv2d(
          1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
    )
    (box_predictor): FastRCNNOutputLayers(
      (cls_score): Linear(in_features=2048, out_features=2, bias=True)
      (bbox_pred): Linear(in_features=2048, out_features=4, bias=True)
    )
  )
)
[10/31 12:34:29 d2.data.build]: Removed 18625 images with no usable annotations. 5390 images left.
[10/31 12:34:29 d2.data.build]: Distribution of instances among all 1 categories:
|  category  | #instances   |
|:----------:|:-------------|
| pneumonia  | 8578         |
|            |              |
[10/31 12:34:29 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
[10/31 12:34:29 d2.data.build]: Using training sampler TrainingSampler
[10/31 12:34:29 d2.data.common]: Serializing 5390 elements to byte tensors and concatenating them all ...
[10/31 12:34:29 d2.data.common]: Serialized dataset takes 1.44 MiB
[10/31 12:34:29 d2.engine.train_loop]: Starting training from iteration 0
[10/31 12:34:45 d2.utils.events]:  eta: 0:06:15  iter: 19  total_loss: 0.3816  loss_cls: 0.1248  loss_box_reg: 0.1951  loss_rpn_cls: 0.03915  loss_rpn_loc: 0.02081  time: 0.7809  data_time: 0.0121  lr: 9.7405e-06  max_mem: 4766M
[10/31 12:35:01 d2.utils.events]:  eta: 0:06:01  iter: 39  total_loss: 0.3343  loss_cls: 0.1074  loss_box_reg: 0.1835  loss_rpn_cls: 0.02785  loss_rpn_loc: 0.0168  time: 0.7858  data_time: 0.0045  lr: 1.9731e-05  max_mem: 4766M
[10/31 12:35:17 d2.utils.events]:  eta: 0:05:45  iter: 59  total_loss: 0.3387  loss_cls: 0.1138  loss_box_reg: 0.1603  loss_rpn_cls: 0.03581  loss_rpn_loc: 0.01446  time: 0.7862  data_time: 0.0040  lr: 2.972e-05  max_mem: 4766M
[10/31 12:35:32 d2.utils.events]:  eta: 0:05:30  iter: 79  total_loss: 0.3965  loss_cls: 0.1326  loss_box_reg: 0.1723  loss_rpn_cls: 0.04455  loss_rpn_loc: 0.01859  time: 0.7860  data_time: 0.0042  lr: 3.9711e-05  max_mem: 4766M
[10/31 12:35:48 d2.utils.events]:  eta: 0:05:14  iter: 99  total_loss: 0.3608  loss_cls: 0.1205  loss_box_reg: 0.1828  loss_rpn_cls: 0.0403  loss_rpn_loc: 0.02028  time: 0.7860  data_time: 0.0040  lr: 4.9701e-05  max_mem: 4766M
[10/31 12:36:04 d2.utils.events]:  eta: 0:04:58  iter: 119  total_loss: 0.3851  loss_cls: 0.12  loss_box_reg: 0.2102  loss_rpn_cls: 0.03687  loss_rpn_loc: 0.01751  time: 0.7862  data_time: 0.0043  lr: 5.9691e-05  max_mem: 4766M
[10/31 12:36:20 d2.utils.events]:  eta: 0:04:43  iter: 139  total_loss: 0.3566  loss_cls: 0.1202  loss_box_reg: 0.184  loss_rpn_cls: 0.02687  loss_rpn_loc: 0.01936  time: 0.7866  data_time: 0.0042  lr: 6.9681e-05  max_mem: 4766M
[10/31 12:36:36 d2.utils.events]:  eta: 0:04:27  iter: 159  total_loss: 0.3938  loss_cls: 0.1092  loss_box_reg: 0.2074  loss_rpn_cls: 0.03628  loss_rpn_loc: 0.02124  time: 0.7868  data_time: 0.0043  lr: 7.9671e-05  max_mem: 4766M
[10/31 12:36:51 d2.utils.events]:  eta: 0:04:12  iter: 179  total_loss: 0.3781  loss_cls: 0.1225  loss_box_reg: 0.2117  loss_rpn_cls: 0.03917  loss_rpn_loc: 0.02111  time: 0.7870  data_time: 0.0043  lr: 8.966e-05  max_mem: 4766M
[10/31 12:37:07 d2.utils.events]:  eta: 0:03:56  iter: 199  total_loss: 0.3099  loss_cls: 0.1152  loss_box_reg: 0.1634  loss_rpn_cls: 0.03182  loss_rpn_loc: 0.01878  time: 0.7869  data_time: 0.0042  lr: 9.9651e-05  max_mem: 4766M
[10/31 12:37:23 d2.utils.events]:  eta: 0:03:40  iter: 219  total_loss: 0.3881  loss_cls: 0.1251  loss_box_reg: 0.2083  loss_rpn_cls: 0.03528  loss_rpn_loc: 0.01547  time: 0.7867  data_time: 0.0041  lr: 0.00010964  max_mem: 4766M
[10/31 12:37:38 d2.utils.events]:  eta: 0:03:24  iter: 239  total_loss: 0.333  loss_cls: 0.1075  loss_box_reg: 0.1769  loss_rpn_cls: 0.02622  loss_rpn_loc: 0.02001  time: 0.7864  data_time: 0.0042  lr: 0.00011963  max_mem: 4766M
[10/31 12:37:54 d2.utils.events]:  eta: 0:03:08  iter: 259  total_loss: 0.3603  loss_cls: 0.1187  loss_box_reg: 0.182  loss_rpn_cls: 0.03413  loss_rpn_loc: 0.01626  time: 0.7864  data_time: 0.0042  lr: 0.00012962  max_mem: 4766M
[10/31 12:38:10 d2.utils.events]:  eta: 0:02:53  iter: 279  total_loss: 0.3467  loss_cls: 0.1135  loss_box_reg: 0.1744  loss_rpn_cls: 0.03746  loss_rpn_loc: 0.01814  time: 0.7866  data_time: 0.0047  lr: 0.00013961  max_mem: 4766M
[10/31 12:38:26 d2.utils.events]:  eta: 0:02:37  iter: 299  total_loss: 0.3719  loss_cls: 0.1154  loss_box_reg: 0.1884  loss_rpn_cls: 0.03224  loss_rpn_loc: 0.02303  time: 0.7868  data_time: 0.0042  lr: 0.0001496  max_mem: 4766M
[10/31 12:38:42 d2.utils.events]:  eta: 0:02:21  iter: 319  total_loss: 0.3641  loss_cls: 0.12  loss_box_reg: 0.1779  loss_rpn_cls: 0.03772  loss_rpn_loc: 0.01872  time: 0.7869  data_time: 0.0042  lr: 0.00015959  max_mem: 4766M
[10/31 12:38:57 d2.utils.events]:  eta: 0:02:06  iter: 339  total_loss: 0.3796  loss_cls: 0.1227  loss_box_reg: 0.2028  loss_rpn_cls: 0.03588  loss_rpn_loc: 0.01835  time: 0.7872  data_time: 0.0043  lr: 0.00016958  max_mem: 4766M
[10/31 12:39:13 d2.utils.events]:  eta: 0:01:50  iter: 359  total_loss: 0.357  loss_cls: 0.1236  loss_box_reg: 0.1965  loss_rpn_cls: 0.03661  loss_rpn_loc: 0.01508  time: 0.7872  data_time: 0.0043  lr: 0.00017957  max_mem: 4766M
[10/31 12:39:29 d2.utils.events]:  eta: 0:01:34  iter: 379  total_loss: 0.337  loss_cls: 0.1093  loss_box_reg: 0.171  loss_rpn_cls: 0.03584  loss_rpn_loc: 0.01315  time: 0.7873  data_time: 0.0042  lr: 0.00018956  max_mem: 4766M
[10/31 12:39:45 d2.utils.events]:  eta: 0:01:18  iter: 399  total_loss: 0.3961  loss_cls: 0.1219  loss_box_reg: 0.1908  loss_rpn_cls: 0.04822  loss_rpn_loc: 0.02207  time: 0.7873  data_time: 0.0040  lr: 0.00019955  max_mem: 4766M
[10/31 12:40:00 d2.utils.events]:  eta: 0:01:03  iter: 419  total_loss: 0.3653  loss_cls: 0.1179  loss_box_reg: 0.1813  loss_rpn_cls: 0.03301  loss_rpn_loc: 0.01778  time: 0.7874  data_time: 0.0041  lr: 0.00020954  max_mem: 4766M
[10/31 12:40:16 d2.utils.events]:  eta: 0:00:47  iter: 439  total_loss: 0.3849  loss_cls: 0.1145  loss_box_reg: 0.1991  loss_rpn_cls: 0.04416  loss_rpn_loc: 0.0179  time: 0.7872  data_time: 0.0041  lr: 0.00021953  max_mem: 4766M
[10/31 12:40:32 d2.utils.events]:  eta: 0:00:31  iter: 459  total_loss: 0.3634  loss_cls: 0.129  loss_box_reg: 0.1881  loss_rpn_cls: 0.03268  loss_rpn_loc: 0.01426  time: 0.7873  data_time: 0.0045  lr: 0.00022952  max_mem: 4766M
[10/31 12:40:48 d2.utils.events]:  eta: 0:00:15  iter: 479  total_loss: 0.3499  loss_cls: 0.1236  loss_box_reg: 0.1678  loss_rpn_cls: 0.04109  loss_rpn_loc: 0.0152  time: 0.7871  data_time: 0.0041  lr: 0.00023951  max_mem: 4766M
[10/31 12:41:04 d2.utils.events]:  eta: 0:00:00  iter: 499  total_loss: 0.3479  loss_cls: 0.1153  loss_box_reg: 0.1894  loss_rpn_cls: 0.03256  loss_rpn_loc: 0.01503  time: 0.7869  data_time: 0.0041  lr: 0.0002495  max_mem: 4766M
[10/31 12:41:04 d2.engine.hooks]: Overall training speed: 498 iterations in 0:06:31 (0.7870 s / it)
[10/31 12:41:04 d2.engine.hooks]: Total training time: 0:06:32 (0:00:00 on hooks)

Inference & evaluation using the trained model

In [75]:
result = evaluate(3000)
In [76]:
df = pd.DataFrame(result)
df.head(10)
Out[76]:
id ground_truth predicted
0 541fd23b-53f2-4bfc-abe6-92f330f2474e 0 1
1 9dcc29e1-433e-426a-8d6e-6243ec981883 0 0
2 1c44e0a4-4612-438f-9a83-8d5bf919cb67 1 1
3 076eab63-d75d-4272-a5d3-13eae02f2c8f 1 1
4 d6ccb839-1988-4113-8233-c7698065a8a1 0 1
5 83126b29-c67f-4296-bf41-882b297c7cac 0 1
6 d877d8be-7d38-43aa-b359-bd86cf3951c3 1 1
7 febbc3ae-c1c8-4625-8cf0-3170c460ca4f 0 1
8 9d982975-b6f9-45ef-893f-4a3e633f9f3b 0 1
9 7d351015-e0a1-4ba9-a310-1b1f8dc72256 0 0
In [77]:
print(classification_report(df['ground_truth'], df['predicted']))
accuracy = accuracy_score(df['ground_truth'], df['predicted'])
print('Model accuracy is: ', accuracy)
              precision    recall  f1-score   support

           0       0.91      0.55      0.68      2047
           1       0.36      0.83      0.50       622

    accuracy                           0.61      2669
   macro avg       0.64      0.69      0.59      2669
weighted avg       0.79      0.61      0.64      2669

Model accuracy is:  0.6129636568002997

Create dataset with image augmentation for positive cases and save

In [29]:
# import json
# out_file = open(os.path.join(data_dir,"coco-full-aug.json"), "w")
# json.dump(get_data_dicts(data_dir+'/train', True), out_file, indent = 4) 
# out_file.close()

Split the dataset into train and validation

In [30]:
with open(os.path.join(data_dir,"coco-full-aug.json"), 'r') as myfile:
    data=myfile.read()

# parse file
obj = json.loads(data)
 
X_train, X_val = train_test_split(obj, test_size=0.1)
print(len(X_train))
print(len(X_val))
25604
2845
In [31]:
train_model()
[10/31 11:49:23 d2.engine.defaults]: Model:
GeneralizedRCNN(
  (backbone): ResNet(
    (stem): BasicStem(
      (conv1): Conv2d(
        3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
        (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
      )
    )
    (res2): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv1): Conv2d(
          64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv2): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
        (conv3): Conv2d(
          64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
      )
    )
    (res3): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv1): Conv2d(
          256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
      (3): BottleneckBlock(
        (conv1): Conv2d(
          512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv2): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
        )
        (conv3): Conv2d(
          128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
      )
    )
    (res4): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
        (conv1): Conv2d(
          512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (3): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (4): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
      (5): BottleneckBlock(
        (conv1): Conv2d(
          1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv2): Conv2d(
          256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
        )
        (conv3): Conv2d(
          256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
        )
      )
    )
  )
  (proposal_generator): RPN(
    (rpn_head): StandardRPNHead(
      (conv): Conv2d(
        1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (objectness_logits): Conv2d(1024, 15, kernel_size=(1, 1), stride=(1, 1))
      (anchor_deltas): Conv2d(1024, 60, kernel_size=(1, 1), stride=(1, 1))
    )
    (anchor_generator): DefaultAnchorGenerator(
      (cell_anchors): BufferList()
    )
  )
  (roi_heads): Res5ROIHeads(
    (pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
      )
    )
    (res5): Sequential(
      (0): BottleneckBlock(
        (shortcut): Conv2d(
          1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
        (conv1): Conv2d(
          1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
      (1): BottleneckBlock(
        (conv1): Conv2d(
          2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
      (2): BottleneckBlock(
        (conv1): Conv2d(
          2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv2): Conv2d(
          512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
        )
        (conv3): Conv2d(
          512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
          (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
        )
      )
    )
    (box_predictor): FastRCNNOutputLayers(
      (cls_score): Linear(in_features=2048, out_features=2, bias=True)
      (bbox_pred): Linear(in_features=2048, out_features=4, bias=True)
    )
  )
)
[10/31 11:49:23 d2.data.build]: Removed 18604 images with no usable annotations. 7000 images left.
[10/31 11:49:23 d2.data.build]: Distribution of instances among all 1 categories:
|  category  | #instances   |
|:----------:|:-------------|
| pneumonia  | 11135        |
|            |              |
[10/31 11:49:23 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
[10/31 11:49:23 d2.data.build]: Using training sampler TrainingSampler
[10/31 11:49:23 d2.data.common]: Serializing 7000 elements to byte tensors and concatenating them all ...
[10/31 11:49:23 d2.data.common]: Serialized dataset takes 1.88 MiB
[10/31 11:49:24 d2.engine.train_loop]: Starting training from iteration 0
[10/31 11:49:40 d2.utils.events]:  eta: 0:06:18  iter: 19  total_loss: 0.4393  loss_cls: 0.1432  loss_box_reg: 0.2186  loss_rpn_cls: 0.0484  loss_rpn_loc: 0.01998  time: 0.7870  data_time: 0.0112  lr: 9.7405e-06  max_mem: 4258M
[10/31 11:49:55 d2.utils.events]:  eta: 0:06:02  iter: 39  total_loss: 0.4202  loss_cls: 0.1366  loss_box_reg: 0.1936  loss_rpn_cls: 0.03301  loss_rpn_loc: 0.01951  time: 0.7894  data_time: 0.0044  lr: 1.9731e-05  max_mem: 4258M
[10/31 11:50:11 d2.utils.events]:  eta: 0:05:47  iter: 59  total_loss: 0.3831  loss_cls: 0.1204  loss_box_reg: 0.1834  loss_rpn_cls: 0.04585  loss_rpn_loc: 0.01743  time: 0.7902  data_time: 0.0042  lr: 2.972e-05  max_mem: 4258M
[10/31 11:50:27 d2.utils.events]:  eta: 0:05:31  iter: 79  total_loss: 0.4094  loss_cls: 0.1199  loss_box_reg: 0.2145  loss_rpn_cls: 0.04436  loss_rpn_loc: 0.02196  time: 0.7904  data_time: 0.0043  lr: 3.9711e-05  max_mem: 4258M
[10/31 11:50:43 d2.utils.events]:  eta: 0:05:16  iter: 99  total_loss: 0.3996  loss_cls: 0.1196  loss_box_reg: 0.2071  loss_rpn_cls: 0.04256  loss_rpn_loc: 0.02356  time: 0.7900  data_time: 0.0039  lr: 4.9701e-05  max_mem: 4258M
[10/31 11:50:59 d2.utils.events]:  eta: 0:05:00  iter: 119  total_loss: 0.3537  loss_cls: 0.1157  loss_box_reg: 0.194  loss_rpn_cls: 0.03908  loss_rpn_loc: 0.01618  time: 0.7902  data_time: 0.0043  lr: 5.9691e-05  max_mem: 4258M
[10/31 11:51:14 d2.utils.events]:  eta: 0:04:44  iter: 139  total_loss: 0.3871  loss_cls: 0.1267  loss_box_reg: 0.1828  loss_rpn_cls: 0.04073  loss_rpn_loc: 0.01428  time: 0.7900  data_time: 0.0043  lr: 6.9681e-05  max_mem: 4258M
[10/31 11:51:30 d2.utils.events]:  eta: 0:04:28  iter: 159  total_loss: 0.3748  loss_cls: 0.122  loss_box_reg: 0.2044  loss_rpn_cls: 0.04132  loss_rpn_loc: 0.01714  time: 0.7899  data_time: 0.0040  lr: 7.9671e-05  max_mem: 4258M
[10/31 11:51:46 d2.utils.events]:  eta: 0:04:12  iter: 179  total_loss: 0.4094  loss_cls: 0.119  loss_box_reg: 0.2154  loss_rpn_cls: 0.04149  loss_rpn_loc: 0.018  time: 0.7899  data_time: 0.0042  lr: 8.966e-05  max_mem: 4258M
[10/31 11:52:02 d2.utils.events]:  eta: 0:03:57  iter: 199  total_loss: 0.3813  loss_cls: 0.1267  loss_box_reg: 0.1877  loss_rpn_cls: 0.04152  loss_rpn_loc: 0.01938  time: 0.7902  data_time: 0.0044  lr: 9.9651e-05  max_mem: 4258M
[10/31 11:52:18 d2.utils.events]:  eta: 0:03:41  iter: 219  total_loss: 0.3865  loss_cls: 0.116  loss_box_reg: 0.1977  loss_rpn_cls: 0.03781  loss_rpn_loc: 0.018  time: 0.7904  data_time: 0.0042  lr: 0.00010964  max_mem: 4258M
[10/31 11:52:34 d2.utils.events]:  eta: 0:03:25  iter: 239  total_loss: 0.3864  loss_cls: 0.1157  loss_box_reg: 0.2014  loss_rpn_cls: 0.03428  loss_rpn_loc: 0.01755  time: 0.7903  data_time: 0.0042  lr: 0.00011963  max_mem: 4258M
[10/31 11:52:49 d2.utils.events]:  eta: 0:03:09  iter: 259  total_loss: 0.3723  loss_cls: 0.1205  loss_box_reg: 0.1892  loss_rpn_cls: 0.04137  loss_rpn_loc: 0.0221  time: 0.7903  data_time: 0.0044  lr: 0.00012962  max_mem: 4258M
[10/31 11:53:05 d2.utils.events]:  eta: 0:02:53  iter: 279  total_loss: 0.4077  loss_cls: 0.1328  loss_box_reg: 0.2031  loss_rpn_cls: 0.05236  loss_rpn_loc: 0.0192  time: 0.7901  data_time: 0.0043  lr: 0.00013961  max_mem: 4258M
[10/31 11:53:21 d2.utils.events]:  eta: 0:02:38  iter: 299  total_loss: 0.4195  loss_cls: 0.1255  loss_box_reg: 0.2  loss_rpn_cls: 0.03802  loss_rpn_loc: 0.02131  time: 0.7901  data_time: 0.0043  lr: 0.0001496  max_mem: 4258M
[10/31 11:53:37 d2.utils.events]:  eta: 0:02:22  iter: 319  total_loss: 0.3758  loss_cls: 0.1171  loss_box_reg: 0.1935  loss_rpn_cls: 0.04375  loss_rpn_loc: 0.0222  time: 0.7899  data_time: 0.0043  lr: 0.00015959  max_mem: 4258M
[10/31 11:53:53 d2.utils.events]:  eta: 0:02:06  iter: 339  total_loss: 0.3554  loss_cls: 0.1079  loss_box_reg: 0.1792  loss_rpn_cls: 0.04105  loss_rpn_loc: 0.02123  time: 0.7899  data_time: 0.0043  lr: 0.00016958  max_mem: 4258M
[10/31 11:54:08 d2.utils.events]:  eta: 0:01:50  iter: 359  total_loss: 0.397  loss_cls: 0.127  loss_box_reg: 0.2092  loss_rpn_cls: 0.03334  loss_rpn_loc: 0.02447  time: 0.7896  data_time: 0.0043  lr: 0.00017957  max_mem: 4258M
[10/31 11:54:24 d2.utils.events]:  eta: 0:01:34  iter: 379  total_loss: 0.3813  loss_cls: 0.1057  loss_box_reg: 0.2066  loss_rpn_cls: 0.03593  loss_rpn_loc: 0.01832  time: 0.7894  data_time: 0.0041  lr: 0.00018956  max_mem: 4258M
[10/31 11:54:40 d2.utils.events]:  eta: 0:01:18  iter: 399  total_loss: 0.339  loss_cls: 0.1019  loss_box_reg: 0.1655  loss_rpn_cls: 0.04117  loss_rpn_loc: 0.01916  time: 0.7895  data_time: 0.0043  lr: 0.00019955  max_mem: 4258M
[10/31 11:54:56 d2.utils.events]:  eta: 0:01:03  iter: 419  total_loss: 0.3669  loss_cls: 0.1246  loss_box_reg: 0.1835  loss_rpn_cls: 0.0291  loss_rpn_loc: 0.01612  time: 0.7895  data_time: 0.0041  lr: 0.00020954  max_mem: 4258M
[10/31 11:55:11 d2.utils.events]:  eta: 0:00:47  iter: 439  total_loss: 0.3947  loss_cls: 0.1278  loss_box_reg: 0.186  loss_rpn_cls: 0.03593  loss_rpn_loc: 0.02088  time: 0.7893  data_time: 0.0043  lr: 0.00021953  max_mem: 4258M
[10/31 11:55:27 d2.utils.events]:  eta: 0:00:31  iter: 459  total_loss: 0.359  loss_cls: 0.1147  loss_box_reg: 0.1762  loss_rpn_cls: 0.03977  loss_rpn_loc: 0.01829  time: 0.7891  data_time: 0.0042  lr: 0.00022952  max_mem: 4258M
[10/31 11:55:43 d2.utils.events]:  eta: 0:00:15  iter: 479  total_loss: 0.3177  loss_cls: 0.1147  loss_box_reg: 0.1554  loss_rpn_cls: 0.03234  loss_rpn_loc: 0.01697  time: 0.7890  data_time: 0.0043  lr: 0.00023951  max_mem: 4258M
[10/31 11:55:59 d2.utils.events]:  eta: 0:00:00  iter: 499  total_loss: 0.4205  loss_cls: 0.1309  loss_box_reg: 0.2085  loss_rpn_cls: 0.03527  loss_rpn_loc: 0.01537  time: 0.7890  data_time: 0.0044  lr: 0.0002495  max_mem: 4258M
[10/31 11:55:59 d2.engine.hooks]: Overall training speed: 498 iterations in 0:06:32 (0.7890 s / it)
[10/31 11:55:59 d2.engine.hooks]: Total training time: 0:06:33 (0:00:00 on hooks)

Inference & evaluation using the trained model on augmented data

In [69]:
result = evaluate(3000)
In [70]:
df_aug = pd.DataFrame(result)
df_aug.head()
Out[70]:
id ground_truth predicted
0 8f54d530-066c-4a62-8d66-10293a8fe7d9 0 0
1 b1572976-1a77-4ed8-a2cd-4c3ee33ff27d 0 0
2 3b81bcea-ee88-4c13-b4f6-2a53161d2b22 0 0
3 f9a571b8-65a5-463c-8ee7-db2f3728f5d2 1 1
4 f2f6d2de-0a65-46f8-8353-3e0304a98732 0 1
In [71]:
print(classification_report(df_aug['ground_truth'], df_aug['predicted']))
accuracy = accuracy_score(df_aug['ground_truth'], df_aug['predicted'])
print('Model accuracy is: ', accuracy)
              precision    recall  f1-score   support

           0       0.86      0.64      0.73      2068
           1       0.43      0.72      0.54       777

    accuracy                           0.66      2845
   macro avg       0.65      0.68      0.64      2845
weighted avg       0.74      0.66      0.68      2845

Model accuracy is:  0.6636203866432338

Make Predictions on Random Data

In [59]:
def get_test_files(dir):
    files = glob.glob(dir+'/'+'*.jpg')
    return list(set(files))
In [68]:
from detectron2.utils.visualizer import ColorMode
# get_test_files(data_dir+'/test')
for d in random.sample(X_val, 10):      
    im = cv2.imread(d['file_name'])
    predictor = DefaultPredictor(cfg)
    outputs = predictor(im)  
    v = Visualizer(im[:, :, ::-1],
                   metadata=pnuemonia_metadata, 
                   scale=0.5, 
                  #  instance_mode=ColorMode.IMAGE_BW   
    )
    out = v.draw_instance_predictions(outputs["instances"].to("cpu")) 
    
    print("\nPrediction")
    cv2_imshow(out.get_image()[:, :, ::-1])

    print("\nGround Truth")
    show(d)
    
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth
Prediction
Ground Truth